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ABSTRACT 

Postoperative adverse events occur all too 
commonly and contribute greatly to our large 
and increasing healthcare costs. Surgeons, as 
well as hospitals, need to know their own 
outcomes in order to recognise areas that need 
improvement before they can work towards 
reducing complications. In the USA, the 
American College of Surgeons National Surgical 
Quality Improvement Project (ACS NSQIP) collects 
clinical data that provide benchmarks for 
providers and hospitals. This review summarises 
the history of ACS NSQIP and its components, 
and describes the evidence that feeding 
outcomes back to providers, along with real-time 
comparisons with other hospital rates, leads to 
quality improvement, better patient outcomes, 
cost savings and overall improved patient safety. 
The potential harms and limitations of the 
program are discussed. 



THE PROBLEM 

Over 30 million operations are performed 
in the USA each year. 1 Postoperative 
adverse events occur frequently. Even in 
common, non-complex cases such as col- 
ectomy (250 000 cases annually), surgical 
site infections occur in 10% of patients. 2 
These adverse events increase hospitalisa- 
tion length and cost. For example, one 
study estimates that the cost of a surgical 
site infection is over US$27 000, 3 while a 
urinary tract infection can cost from US 
$675 to US$2800. 4 A single case of 
ventilator-associated pneumonia can add 
US$50 000 to the cost of an admission. 5 6 
Length of stay increases when complica- 
tions occur, with an extra 3-11 days in 
hospital required for respiratory events. 7 



Payments are being reduced for some 
adverse events, such as central line infec- 
tions and surgical site infections. The 
Affordable Care Act of 2012 established 
the Hospital Readmission Reduction 
Program and in turn the Centers for 
Medicare and Medicaid Services began 
implementing a process to reduce pay- 
ments for certain 30-day readmissions. 

The largest and best known interven- 
tion for measuring and reporting surgical 
outcomes in the USA is the American 
College of Surgeons National Surgical 
Quality Improvement Project (ACS 
NSQIP). Risk-adjusted postoperative out- 
comes reported by NSQIP provide bench- 
marks intended to spur local quality 
improvement efforts to produce better 
patient outcomes. This review assesses the 
evidence to determine whether ACS 
NSQIP has improved surgical care and 
patient safety. 



PATIENT SAFETY STRATEGIES 

The multi-component ACS NSQIP in part 
grew out of efforts initiated by Veterans 
Affairs (VA) health system researchers and 
clinicians in the late 1980s. In response to 
concerns about high complication rates in 
VA hospitals, VA NSQIP was officially 
launched in 1994 to collect and report 
clinical variables and outcomes across all 
VA hospitals. 8 9 Another factor contribut- 
ing to ACS NSQIP was the success of pro- 
grams like the New York State Cardiac 
Surgery Reporting System (CSRS) in the 
1990s. 10 Basic concepts of the New York 
State CSRS have been adopted by other 
states (California, Pennsylvania) and have 
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spread across the USA through the efforts of the 
Society of Thoracic Surgery (STS) Registry, which 
incorporated public reporting. The measurement and 
public reporting of cardiac surgery outcomes has also 
spread to England. 11 

With growing attention being paid to systematic 
reporting of surgical outcomes, non-V\ hospitals 
became interested in applying the VA experience to 
their data reporting and quality improvement pro- 
grams in the late 1990s. A pilot study in three civilian 
hospitals (University of Michigan, Emory University 
and University of Kentucky) demonstrated the feasibil- 
ity of this initiative in the private sector. 12 Following 
this pilot, the ACS took the lead to expand efforts to 
a broader group of hospitals (14 sites) in 2001, and 
the formal ACS NSQIP began in 2004. 13 An overview 
of the key aspects of the program are provided in 
figure 1. 

The intervention consists of several basic compo- 
nents. First, a surgeon champion (an individual staff 
member at the participating site) assumes a significant 
role in establishing and overseeing the participation of 
each site. Second, a surgical clinical reviewer (SCR) is 
trained to collect data on preoperative clinical vari- 
ables and on 30-day outcomes. The third component 
is the development of risk-adjusted models of 
expected mortality and morbidity by procedure type. 
Fourth, data are analysed and presented back to the 
individual sites alongside masked data for the other 
sites. Data are provided for each of 21 morbidities 
(such as unplanned readmissions, urinary tract infec- 
tions and pulmonary embolism) in addition to mortal- 
ity. Sites are displayed as being high (worse than 
expected) or low outliers (better than expected) for 
each category of morbidity and for mortality. Lastly, 
the participating sites are encouraged to address and 
correct problem areas of clinical care. 

Originally, ACS NSQIP reported observed-to- 
expected (O/E) 30-day mortality and morbidity ratios. 
Over the past 8 years, risk adjustment and modelling 



has evolved to improve the accuracy of the reported 
outcomes. 14 Recently, the system has been adapted to 
better account for surgical case complexity by introdu- 
cing a procedure mix adjustment. Additionally, 
ACS NSQIP no longer reports O/E ratios but instead 
provides logistical ORs using hierarchical modelling 
with shrinkage adjustment (figure 2). This allows for 
more stable estimates for small sample sizes, such as 
lower volume procedures or smaller hospitals. 
Additionally, this allows for more valid and accurate 
reporting of individual provider rates. Currently, both 
facility and surgeon data are collected. Individual 
facility results are provided routinely, but those for 
individual surgeons are provided upon request. 

From reports generated by ACS NSQIP, institutions 
can see where they are a high outlier for certain pro- 
cedures and identify where improvement is needed. 
An example of a de-identified report is provided in 
figure 2. Auditing by ACS NSQIP staff occurs ran- 
domly and for cause, that is if a site reported results 
too good to be true, and an audit for cause revealed 
upcoding of comorbidities and downcoding of 
complications or dramatic changes in results. The 
audits are also measures of data collection quality. 
Program audits reported low disagreement rates 
(between the auditor and site data collector) rela- 
tively early in program development (3.15% in 
2005) and this rate has continued to drop (1.56% 
in 2008). 15 

While the responsibility for making changes 
remains with the individual sites, the administrative 
ACS NSQIP body provides support in terms of case 
report of successful interventions, provision of best 
practices, national meetings, and monthly supportive 
conferences calls with the surgeon champions and 
clinical reviewers. For example, ACS NSQIP staff will 
reach out to individual sites to assist them by connect- 
ing them with others who have made improvements 
in the area of concern or interest. Additionally, the 
annual ACS NSQIP meeting serves as a critical forum 



Prospectively or concurrently collects clinical data 

Peer-controlled database as it was designed by and is regulated by surgeons.* 
Generates risk-adjusted assessments of outcomes 
Results are fed back to the hospitals and surgeons 

Provides bench-marks by giving comparison 30-day outcomes from the other 
participating sites 

Routine and spot audits to check data integrity 

Figure 1 Key aspects of the American College of Surgeons National Surgical Quality Improvement Project (ACS NSQIP). *Surgeons 
selected the content, make decisions about changes in the way data are collected and analysed, and they provide oversight. 
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General/Vascular 



01/01/11-12/31/11 

0.78 1.01 1.00 



Report /Site ID: 0000/0000 




GV GV GV Cardiac GV GV GV GVDVT/PE GV Renal GVUTI GV SSI GV ROR 

Mortality Morbidity Pneumonia Unplanned Ventilator Failure 

intubation >48 hrs 

Figure 2 Example reporting events by odds ratio using hierarchal modelling with shrinkage adjustment. Hospital specific bar plots. 
Each box shows the distribution of odds ratios for hospitals in the model; the top and bottom of each box give the smallest and 
largest values, and horizontal lines give the deciles. The point and vertical line within each box give the individual hospital's odds ratio 
and 99% CI. General and vascular surgery (GV) outcomes models are reported. DVT, deep venous thrombosis; PE, pulmonary 
embolism; ROR, return to operating room; SSI, surgical site infection; UTI, urinary tract infection. Adapted from Cohen, JACS, 201 3. 14 



for sites to present and discuss a range of quality 
improvement efforts and successes. 

REVIEW PROCESSES 

A Medline search up to November 2012 carried out 
using the search terms NSQIP and National Surgical 
Quality Improvement Project identified 169 studies. 
The review also included the Cochrane Registry and 
gray literature available on the ACS NSQIP website 
(http://www.acsnsqip.org/). Following title and abstract 
screening, the author selected any article that pro- 
vided evidence for the benefits or harms of VA 
NSQIP or ACS NSQIP, or quantitative information 
describing how ACS NSQIP was implemented. There 
was no restriction on study design. 

The author supplemented this literature review with 
interviews with the leadership and administrators in 
ACS NSQIP, as well as four surgeon champions. These 
surgeons were selected to represent different partici- 
pants in the program: community, academic, leader- 
ship and a state-wide collaborative. The interviews 
explored weaknesses as well as strengths of the 
program. 

BENEFITS AND HARMS 

Benefits 

AVA NSQIP review of over 400 000 cases performed 
between 1991 and 1997 showed that 30-day mortality 
and morbidity rates for major surgery fell by 9% and 
30%, respectively. 16 Reductions in one postoperative 
complication alone, surgical pneumonia, are estimated 
to have saved the VA US$9.3 million annually, and the 



overall reduction in morbidity may have saved billions 
since the program was started. 8 17 18 

Two published longitudinal studies reached diver- 
gent conclusions on the effects of reporting in ACS 
NSQIP They first looked at changes from 2005 to 
2007 in ACS NSQIP-participating sites (N=183) for 
all outcomes measured and surgical specialties using 
risk adjustment and accounting for hospital procedure 
volume. 19 For the most recent period of 2006-2007, 
118 hospitals participated long enough to produce 
clinically useful data. The authors found that 82% of 
hospitals had improved morbidity and 66% had 
improved mortality. The adjusted absolute difference 
in O/E ratio was -0.114 for morbidity and -0.174 
for mortality (negative numbers indicate less morbid- 
ity and mortality). Similar results were seen when the 
researchers accounted for institutional volume. They 
also found that the number of high outliers (those 
with worse outcomes) decreased over time and the 
number of low outliers (those with better outcomes) 
increased. Institutions with high outliers were more 
likely to improve and had larger mean changes in out- 
comes. It was estimated that an average of 200-500 
complications and 12-36 deaths may have been 
avoided. 19 

The other study compared ACS NSQIP to a private 
sector collaborative based at the University of 
Michigan Medical Center. 20 The Michigan Surgical 
Quality Collaborative (MSQC) includes 34, mainly 
community (68%) hospitals, unlike the ACS 
NSQIP-participating hospitals, which are primarily 
academic/teaching institutions. Sixteen MSQC 
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hospitals were assessed over two time periods and 
compared to the 126 non-Michigan NSQIP hospitals 
over the same time periods. MSQC hospitals had a 
decrease in morbidity from 10.7% to 9.7% (9.0% 
reduction, p=0.002; OR 0.898) over 3 years, whereas 
morbidity did not change for the ACS NSQIP hospi- 
tals in either time period or between the periods 
(12.4%; OR 1.0). 20 

The potential impact of participating in ACS NSQIP 
on complication rates and mortality has been reported 
by individual hospitals and collaboratives. Although 
improvements in morbidities have been large, mortal- 
ity changes have ranged from none to modest. 
Mortality for most general and vascular surgical pro- 
cedures is generally low, leaving little room for 
improvement. In contrast, improvements in morbid- 
ities are commonly reported. One local initiative 
showed that targeted efforts reduced respiratory com- 
plications and that after 7 months the rate of post- 
operative pneumonia had been reduced to zero from a 
peak monthly rate of 2%. 21 Another state collabora- 
tive showed improvements in a range of postoperative 



events. However, most of the reports of improve- 
ment in single institutions or later collaboratives have 
been presented in forums other than peer-review 
publications. 

At the July 2011 ACS NSQIP national meeting, 20 
presentations reported reductions in morbidity follow- 
ing an intervention. In all these cases, ACS NSQIP 
data enabled the hospitals to target an area with 
worse-than-expected outcomes and to intervene, with 
resulting improvement (table 1). These presentations 
were selected as representative of successful quality 
improvement efforts that developed as direct 
responses to outcomes shown to be in need of 
improvement. These collective results demonstrate 
how varying sites and collaboratives worked to make 
specific changes to improve outcomes. 

Almost all these studies have a pre-post design and 
therefore have all the accompanying limitations, 
including regression to the mean. However, the results 
are unlikely to be explained solely by regression to the 
mean, primarily because of the size of the effect and 
the post-intervention result. Additionally, these 



Table 1 Example of interventions and changes in outcomes in ACS NSQIP hospitals/collaboratives 



Hospital 



Complication 



Intervention 



Outcome 



Hershey Medical 
Center, Penn 
State 35 

University of 
Virginia 

Massachusetts 
General Hospital 37 



Hospital A 

Hospital B 
Hospital C 

Hospital D 



Hospital E 



19.3% SSI in patients with 
diabetes; 8% in patients without 
diabetes 

VTE 3.4% (2008) 

17.6% SSI (national average 
8.1%) in colorectal resections, 
high BMI was a risk factor 

Vascular surgery morbidity 0/E 
ratio 1.19, (99% CI 0.93 to 
1.48) 

UTI rate 7.0% vs 4.7% 
(p<0.087) 

Identified a rise in organ space 
infections 



VTE 17.6% 



Glucose control protocol 

VTE risk assessment and order set 



Protocol for wound wicking for BMI >25 kg/m 2 , 
SCIP measures, glycaemic control 

Physician order entry templates, Foley catheter 
removal algorithm, silver-coated catheters for 
selected patients, identify procedures not 
requiring a catheter, educational campaign for 
clinicians 

Standardised orders, proper antibiotic use, 
morbidity conference presentations, skin 
preparation changes 



Risk stratification, best practices, standardised 
orders 



Unplanned reintubation 3% (0/E Tracking tool, risk assessment, improved 
1.56) pulmonary hygiene intervention 

Ventilator>48 h 3.84% (0/E 
1.71) 

Ventilator use for >48 h 2.24% Tracking tool, standardised orders, patient 



(0/E 1.7) 



Overall orthopaedic DVT rate 
3.1% 

Knee arthroplasty DVT rate 
10.1% 



education 



Identified variations in DVT prophylaxis practice, 
surgeon-specific review, standardised care 



Reduction of SSI 0/E from 1.31 to 0.78 
Reduction of VTE rate from 3.4% to 0.2% 
(2008-2009) 

Reduction of SSI from 17.6% to 1 1.2% 

(36% reduction) 

(2003-2006) 

Reduction of UTI from 7.0% to 1.8% 
Morbidity 0/E ratio decreased from 1.19 
(99% CI 0.93 to 1.48) to 0.93 (99% CI 
0.67 to 1.48) (76% reduction) 
(2003-2004) 

Organ space infection increase attributed to 
increased leak rates and identified surgical 
technique issues; improvements seen, but 
rate still high 
(2005-2010) 

VTE decreased from 17.6 to 2.3%; 0/E 
decreased from 1.88 to 1.05 
(2006-2010) 

TBD 



Ventilator use for >48 h decreased from 
2.24% to 1.19% 
(0/E 1.7 to 0.83) 
(2008-2010) 

Reduction of overall orthopaedic DVT rate 

from 3.1% to 1.1% 

Reduction of knee arthroplasty DVT rate 

from 10.1% to 1.6% 

(2008-2010) 



Hospitals A-E are representative examples taken from the ACS NSQIP data portal website, accessed 13 December 201 1. Reprinted by permission of 
American College of Surgeons NSQIP. 

ACS NSQIP, American College of Surgeons National Surgical Quality Improvement Project; BMI, body mass index; DVT, deep venous thrombosis; SCIP, 
Surgical Care Improvement Project; SSI, surgical site infection; TBD, to be determined; UTI, urinary tract infection; VTE, venous thromboembolism. 
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changes occur in response to directed efforts for par- 
ticular areas, such as decreasing deep venous throm- 
bosis (DVT) rates or reintubation rates. Furthermore, 
Hall et al 23 demonstrated that both bad (high outlier) 
and good (low outlier) hospitals improve. Taking into 
consideration the large body of pre-post work, which 
shows repeatedly each year of the program, support 
that directed efforts were largely responsible for the 
reported improvements. 

The way NSQIP improves care is multifaceted. To 
improve care and reduce complications, surgeons 
must know the outcomes of their own procedures. 
The data must be of high quality and reliable, and risk 
adjustment must be adequate to allay concerns about 
comparing 'apples to oranges'. This comparison 
allows surgeons and hospitals to see how they 
compare in terms of outcomes, which promotes 
accountability and stimulates work to correct the pro- 
blems. Most sites (59% of those surveyed) were 
unaware of their hospital's adverse event rates, 
let alone how they compared to other hospitals, until 
after they enrolled in ACS NSQIE 3 

A particular feature of the program is its use of 
detailed clinical data collected from the medical 
records. A study comparing administrative and claims 
data collected by the University Health System 
Consortium (UHC) program showed that ACS NSQIP 
identified a greater number of complications (61%) 
than UHC, including 97% more surgical site infec- 
tions and 100% more urinary tract infections. 24 

Furthermore, ACS NSQIP identifies adverse events 
following discharge. Studies using ACS NSQIP show 
that more than 50% of complications occur after dis- 
charge. For colectomies, 45% of deep surgical site 
infections, 39% of organ space infections and 28% of 
DVT occur after patients have left hospital. 25 
Identifying complications that occur outside the hos- 
pital is the prerequisite first step to developing 
changes in care to help prevent them. 19 

Harms 

Few published studies have assessed the potential and 
actual harms of this program, and most are specula- 
tive. A primary concern has been that surgeons will 
avoid high-risk cases for fear of adversely affecting 
their O/E outcomes assessments. This issue was raised 
early in the process of implementing report cards 
when anecdotal evidence appeared to suggest that as 
the result of implementing the New York CSRS, high- 
risk coronary artery bypass graft (CABG) patients 
were being diverted instead to the Cleveland Clinic. 26 
However, subsequent and more comprehensive ana- 
lyses could not document any systematic exclusion of 
high-risk patients from CABG operations, and showed 
that, on the contrary, the severity of illness and 
comorbidities of operated patients has increased over 
the years. 27 28 The longitudinal ACS NSQIP study 
also supported this finding, showing that the risk 



profile and illness severity for surgical patients has 
increased over time. 19 Another concern is that the 
outcomes for outpatient cases or for a hospital or 
surgeon who performs a small volume of procedures 
might need longer follow-up, possibly for more than 
a year, to accurately assess quality. 29 The question 
has been raised that surgeons could alter treatment 
plans for patients based on individual operative risk 
rather than giving the patient the option of a proced- 
ure with a potentially better long-term functional 
outcome. A theoretical example would be in vascular 
surgery, where a high-risk patient eligible for a distal 
bypass would be recommended an amputation 
instead. 

IMPLEMENTATION CONSIDERATIONS AND COSTS 

Implementation context 

The program requirements include site administrative 
support, a surgeon champion, and participation in a 
series of conference calls and the national ACS NSQIP 
meeting. Data reporting is mandated to follow par- 
ticular rules, such as accrual of particular data and 
30-day follow-up information. ACS NSQIP personnel 
perform audits to help maintain data quality. For 
small hospitals, the effort and cost may be less than 
for larger facilities, depending on the volume of cases. 

ACS NSQIP has been implemented in a variety of 
settings including large academic hospitals, smaller 
community hospitals and large and small state-wide 
consortia. It soon became apparent that a variety of 
program models were needed to accommodate differ- 
ing clinical volumes. Program options vary in terms of 
number of variables collected, surgical specialty, if 
procedures are specifically targeted, and case sampling 
required. Currently, more than 500 sites are enrolled 
in ACS NSQIP, which represents roughly 10% of the 
almost 4500 hospitals in the USA. The distribution of 
the more than 525 sites that reported clinically useful 
data is shown in the map in figure 3 . 

Overall, 49% of participating sites are teaching or 
academic centres. The majority of these hospitals are 
high volume, as only 3% perform less than 100 proce- 
dures per year, 7% perform 100-299 procedures per 
year, 43% perform 300-499 procedures per year, and 
47% perform more than 500 procedures per year. 
This skewed distribution of hospital size means that 
the 10% of hospitals participating in ACS NSQIP rep- 
resent 32% of the procedures performed. 30 Certain 
complex procedures are captured at an even higher 
rate, for example, 57% of oesophagectomies and 53% 
of pancreatectomies billed to Medicare are performed 
at ACS NSQIP sites (table 2). 

Collaboratives are a main feature of ACS NSQIP 
and have taken many different forms (some are a 
handful of geographically close hospitals while others 
are all hospitals in a state) that work as a team to 
implement the program and quality improvement. 
They also can represent a disease or patient 
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Number of Sites by State, Region, and Country Included in the January 2013 SAR (358) 

ACS NSQIP Adult 




Figure 3 Geographical distribution of American College of Surgeons National Surgical Quality Improvement Project (ACS NSQIP) 
participating sites. Number of sites by state, region and country included in the January 2013 NSQIP semi-annual report. Reprinted by 
permission of ACS NSQIP. 



population and so members of a collaborative need 
not be geographically close together. In addition to 
providing intellectual and practical support to each 
other for quality improvement initiatives, they provide 
a collective voice for bargaining with potential sources 
of funding. One reported approach is for the main 
insurer for the hospitals in the collaborative to pay 
50% of the cost of the program over a set number of 
years. Sometimes an option to renew the financial 
support is given if certain milestones are met. Some 
payors have judged there is a business case for helping 
support ACS NSQIP participation because of per- 
ceived cost savings. Table 3 shows the current list of 
active collaboratives. 

A pilot paediatric ACS NSQIP collaborative collects 
data for patients under 18 years of age. 31 32 Variables 
have been modified for paediatric surgery practices 
and needs. 

There are many examples of different collabora- 
tives, along with unique challenges facing different 
hospital types. The hospitals proposing to develop a 
collaborative negotiate as a group for financial 
support from a variety of sources, shape the program 
for their own needs, and work together to make 
quality improvement changes. Two examples, one of a 
community of hospitals starting a small state-wide col- 
laborative (Tennessee Surgical Quality Collaborative, 
TSQC) and another where state-wide participation in 
a collaborative was developed (Florida Surgical Care 
Initiative, FSCI), are described in detail below. 



In 2004, after being introduced to the recently 
started ACS NSQIP at the national ACS meeting, a 
member of a community hospital in Tennessee 
initiated a process that resulted in a collaborative that 
included hospitals, payors and the Tennessee Hospital 
Association. The collaborative would be controlled by 
a leadership committee that comprised four surgeons 
appointed by the local society chapter, along with 
hospital CEOs and a member of the Tennessee 
Hospital Association. The proposal included funding 
for the participation of eight hospitals, estimated to 
be US$2 550 000 for 3 years. The money covered half 
the expense of the SCR, salary support for the 
surgeon champions, and administrative costs. This 
example highlights many of the key components for 
building a successful program — surgical leaders taking 
a role, supportive administration, and collaboration 
with other hospitals. 

A strikingly different collaborative was set up in 
Florida. The Florida Hospital Association (FHA) was 
aware of the high surgical mortality demonstrated by 
the Dartmouth Atlas project (which identified dispar- 
ities in access to and utilisation of health care) in their 
state. 33 The FHA, along with the payor, BlueCross 
BlueShiled, collaborated to generate a financial incen- 
tive for hospital participation. A new version of the 
program collecting only four outcomes was devel- 
oped, thus lowering costs. Currently, 64 hospitals are 
participating in the FSCI and the participation of 39 
more is pending. This example demonstrates 
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Table 2 Percentage of Medicare surgical cases covered by ACS NSQIP 



Procedure! 


MC cases 

in MCniD 


MC cases not 

in MCniD 

in IMDlJIr 


Total MC 

cases 


Percentage covered 

Dy IMjvIt 


Oesophagectomy 


1158 


875 


2033 


57.0% 


Cystectomy 


3346 


4501 


7847 


42.6% 


Abdominal aortic aneurysm repair 


3762 


6448 


10210 


36.8% 


Pancreatectomy 


3901 


3399 


7300 


53.4% 


Colectomy 


32 444 


103 056 


135 500 


23.9% 


Proctectomy 


6745 


15 767 


22 512 


30.0% 


Aortoiliac bypass 


2255 


4974 


7229 


31.2% 


Lower extremity bypass 


12 203 


30 100 


42 303 


28.8% 


Liver resection 


2465 


2201 


4666 


52.8% 


Hip fracture repair 


40 030 


151 140 


191 170 


20.9% 


Abdominoplasty 


1058 


1829 


2887 


36.6% 


Lung resection 


16 065 


27 391 


43 456 


37.0% 


Endovascular abdominal aortic aneurysm repair 


8944 


17 324 


26 268 


34.0% 


Nephrectomy 


9727 


16 375 


26 102 


37.3% 


Hysterectomy 


17 954 


45 108 


63 062 


28.5% 


Total hip arthroplasty 


56 700 


195 528 


252 228 


22.5% 


Laminectomy 


60 650 


154 858 


215 508 


28.1% 


Transurethral resection of the prostate 


11 345 


42 928 


54 273 


20.9% 


Ventral hernia 


19 360 


57 735 


77 095 


25.1% 


Carotid endarterectomy 


20 588 


59 710 


80 298 


25.6% 


Total knee arthroplasty 


72 916 


279 642 


352 558 


20.7% 


Prostatectomy 


10 677 


18 808 


29 485 


36.2% 


Breast reconstruction 


455 


700 


1155 


39.4% 


Appendectomy 


8802 


31 635 


40 437 


21.8% 


Thyroid 


5358 


12 598 


17 956 


29.8% 


Gastrectomy 


3782 


7382 


11 164 


33.9% 


Carotid stent 


3648 


7883 


11 531 


31.6% 


Small bowel resection 


10 784 


30 836 


41 620 


25.9% 


Mastectomy 


6417 


21 378 


27 795 


23.1% 


Cholecystectomy 


29 386 


1 17 327 


146 713 


20.0% 


Total 








32.0% 



ACS NSQIP, American College of Surgeons National Surgical Quality Improvement Project; MC, Medicare. 



additional features that help encourage participation: 
individuals at the state level and hospital administra- 
tion taking a lead, a flexible program designed to fit 
the needs of the collaborative, and the role of the 
local payor in incentivising hospital participation. 



Costs 

The costs of participation vary depending on what 
type of program the hospital joins. The annual admin- 
istrative fee varies by hospital size and level of partici- 
pation, salary of the SCR, and optional bonus 
payments to support the surgical champion or quality 
improvement team. This fee ranges from US$10 000 
(rural and hospitals that deal with <2000 cases/year) 
to US$25 000 (>2000 cases). Hospitals can lower 
their costs by participating in a collaborative. 

The salary for the SCR comprises the bulk of the 
expense of participation. Previously, the clinical 
reviewer had to be a registered nurse (RN), but 



because of issues such as nursing shortages, indivi- 
duals such as licensed vocational nurses, with medical 
training but without advanced nursing degrees, have 
been successfully employed as SCRs. All reviewers 
must pass a credentialing examination annually. 
Their expenses will vary based on experience, level 
of training and region, and range from around US 
$40 000 per year to US$100 000 for an experienced 
RN. Many hospitals suggest that paying for a 
surgeon champion (an amount such as US$5000) is 
helpful for increasing their involvement, although a 
recent survey of surgical champions (109 respon- 
dents) found that 72.5% did not receive salary 
support compensation. 34 

The highest total cost of participation is estimated 
to be US$135 000 annually; however, this estimate is 
for a large hospital that hires an RN as the 
reviewer. 35 36 Many participating hospitals may pay 
less as they have lower volumes of patients and there- 
fore decide to participate in a smaller program. 



Maggard-Gibbons M. BMJ Qual Saf 2014;23:589-599. doi: 10.11 36/bmjqs-20 1 3-002223 



595 



Narrative review 



Table 3 List of ACS NSQIP collaboratives including type, number of sites and payor 

Number 



Group 


Tvpe 


of sites 


Payor involvement 


Canadian National Surgical Quality Improvement Collaborative (CAN-NSQIP) 


Regional 


r 
0 


Canadian health authorities 


Connecticut Surgical Quality Coalition (CTSQC) 


Regional 


5 


None at this time 


Department of Defense/TRICARE 


System-wide 


16 


Department of Defense/TRICARE 


Florida Surgical Care Initiative (FSCI) 


Regional 


63 


BlueCross BlueShield of Florida 


Fraser Health Systems (Canada) 


System-wide 


3 


Fraser Health Authority 


Illinois Surgical Quality Improvement Collaborative (ISQIC) 


Regional 


12 


None at this time 


Kaiser Permanente Northern California Regional NSQIP Collaborative 
(KPNCRNC) 


System-wide 


21 


Kaiser Permanente Northern California 


Kaiser Permanente Southern California Regional NSQIP Collaborative 

(\fDk\rQK\r\ 
[Kr IMLKIMLj 


System-wide 


8 


Kaiser Permanente Southern 
California 


MaineHealth Collaborative 


System-wide 


b 


MaineHealth 


Mayo Clinic Surgical Quality Consortium (MCSQC) 


System-wide 


5 


Mayo Clinic 


Northern California Surgical Quality Collaborative (NCSQC) 


Regional 


4 


None at this time 


Nebraska Collaborative 


Regional 


2 


BlueCross BlueShield of Nebraska 


Oregon NSQIP Consortia 


Regional 


8 


None at this time 


Pennsylvania NSQIP Consortia 


Regional 


10 


None at this time 


Partners Healthcare 


System-wide 


5 


BlueCross BlueShield of 
Massachusetts 


Surgical Quality Action Network — British Columbia, Canada (SQAN) 


Regional 


21 


BC Patient Safety and Quality Council 


Tennessee Surgical Quality Collaborative (TSQC) 


Regional 


10 


BlueCross BlueShield of Tennessee 
Health Foundation 


Upstate New York Surgical Quality Initiative 


Regional 


7 
/ 


Excellus 


ALb Nbijir LoiecTomy LoiiaDoraiive 


\ / i r+- 1 i -. 1 

virtual 


DO 


None at this time 


ACS NSQIP Glucose Control Collaborative (Pending) 


\ / 1 1-+ 1 i -i 1 

virtual 


A 

4 


None at this time 


ALb NbUir Kurai LoiiaDorative vr ending; 


\ / 1 1-+ 1 i -i 1 

virtual 


r 
J 


None at this time 


ACS NSQIP Residency Training Collaborative (Pending) 


\ / 1 1-+ 1 i -i 1 

virtual 


td n 
I dU 


None at this time 


Indiana Collaborative (Pending) 


Regional 


7 


None at this time 


Maryland Collaborative (Pending) 


Regional 


3 


None at this time 


Texas Collaborative (Pending) 


Regional 


16 


None at this time 


Virginia Collaborative (Pending) 


Regional 


11 


None at this time 


Wisconsin Collaborative (Pending) 


Regional 


6 


None at this time 



Adapted from ACS NSQIP Annual Meeting, July 201 1. Reprinted by permission of American College of Surgeons NSQIP. 
ACS NSQIP, American College of Surgeons National Surgical Quality Improvement Project; TBD, to be determined. 



Since the overarching goal of ACS NSQIP is to 
reduce complications, which are costly, the business 
case for participating is that the expense of the 
program translates into savings for the hospital. 
Examples of such savings reported by NSQIP sites are 
shown in table 4. Pre-post data without control 
groups are shown and so inference of a causal rela- 
tionship is limited by the study design. 

One cost-effectiveness study of ACS NSQIP has 
been published. Costs and outcomes for 2229 general 
and vascular surgery cases at one large academic study 
were assessed. The study compared two time intervals 
- one 6 months and the other 1 year from inception 
of the program. Postoperative events declined over 
time, from 17 to 13 percent. The incremental costs 
were $832 and $266 per patient for the two time 
periods, meaning the cost per patient of the program 
declined after the first 6 months of participation. The 
incremental cost-effectiveness ratio to avoid 1 



postoperative event was $25,471 in the first 6 
months, declining to $7,319 in the second time 
period, meaning that the longer the institution partici- 
pated in the program, the more cost-effective the 
program became. 37 

DISCUSSION 

Although no randomised trials have assessed the use 
of outcomes measurements and reporting in surgery, 
the strength of the evidence that doing so improves 
operative mortality and morbidity is moderate or even 
high, given the strong theoretical rationale why it 
should work, the evidence that outcome reporting has 
likely improved surgical outcomes in other settings 
(eg, the New York state CSRS), the numerous reports 
from ACS NSQIP sites describing the implementation 
of quality and safety initiatives following identification 
of high outlier status, and the ensuing, sometimes dra- 
matic, improvements in those outcomes. There is a 
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Table 4 Published examples of changes in complications and costs following participation in ACS NSQIP 



Hospital 


Complication reduction 


Savings 


Surrey Memorial 
Hospital 32 


Reduced SSI over four years: from 13%, to 10%, 
to 7.5%, to 7.2% 


US$2.54 million in savings 


Henry Ford Hospital 36 


Reduced LOS by 1.54 days over 4 years for general 
surgery, vascular and colorectal procedures 


US$2 million in annual savings (increased billing by US$2.25 
million/year as underbillings were identified) 


VA 8 


Surgical pneumonia alone 


US$9.3 million in savings annually 


University of Michigan 
Medical Center 5 


Respiratory complication 


US$51 409 per event; a reduction of two such complications 
per year pays for participation 


Hershey Medical Center, 
Penn State 33 


Additional cost attributable to a postoperative complication 
is US$16 371 


Avoiding one postoperative complication equals a cost saving 
of US$9052 



ACS NSQIP, American College of Surgeons National Surgical Quality Improvement Project; LOS, length of stay; VA, Veterans Affairs. 



great deal of experience on how to implement ACS 
NSQIP as it has been rolled out in more than 400 hos- 
pitals. Some of the key components of ACS NSQIP 
(collecting complications data, sharing models of O/E 
results, multi-site data collection systems across institu- 
tions that provide results back to the sites for bench- 
marking, contexts for learning and sharing tools that 
appear to be effective across sites) are similar to those 
of other successful patient safety practices such as the 
Michigan Keystone ICU Project to reduce catheter- 
related bloodstream infections. 38 Despite ACS NSQIP 
and the Keystone ICU Project having started with dif- 
ferent original 'interventions' (the feedback of 
procedure-specific surgical outcome data to surgeons 
and a checklist of processes to reduce infections), the 
fact that current versions of the interventions include 
many similar components suggests the implementation 
of certain types of practices across hospitals is 
generalisable. 

ACS NSQIP provides hospitals and providers with 
usable clinical data that are otherwise not available to 
them. Currently, all hospitals use administrative data 
to some degree to assess quality through the CMS 
Hospital Compare program or the Surgical Care 
Improvement Project (SCIP). These data lack clinical 
information and are limited by the variables reported 
for claims. More importantly, the correlation between 
administrative data and actual complications or diag- 
noses is inadequate. ACS NSQIP uses detailed clinical 
data to highlight areas where improvements are 
needed. 

The greatest benefit has been seen in the larger hos- 
pitals in the procedures with higher complication 
rates. Whether the above improvements will transfer 
to low-risk but common procedures, such as out- 
patient procedures, is unclear. Most of the early adop- 
ters have been large academically affiliated hospitals. 
How successfully and widely the program can be 
implemented at smaller hospitals remains to be seen. 
ACS NSQIP has been flexible in terms of changing to 
fit the needs of those participating. The program spe- 
cifics have been adapted for variable hospital sizes and 
large versus small collaboratives, and the program has 



provided vastly different modules for specific proce- 
dures, and even a new program for a specific surgical 
population — ACS NSQIP Pediatric for paediatric 
surgery reporting. 

NSQIP was developed specifically for surgery. Like 
other surgical outcomes reporting systems, the con- 
ceptual model behind ACS NSQIP works best in situa- 
tions where outcomes are measureable within a short 
time frame after the relevant care has been delivered, 
and there are reasonable means to adjust for case-mix 
differences. This model may not generalise to all types 
of health care, such as primary care, chronic care etc, 
but may translate well to those sharing similar clinical 
properties, like treatment provided in intensive care 
units. 

A limitation of ACS NSQII^ or any outcomes-based 
quality improvement program, is that knowing out- 
comes does not necessarily provide the answer to pro- 
ducing better outcomes, rather, it requires the unit to 
know itself and identify the defect. ACS NSQIP does 
provide educational tools, such as guidelines and best 
practices, but it still requires leadership at the local 
level to lead the charge. Lastly, as process improve- 
ment is needed to sustain better outcomes, it is also 
important to measure and benchmark process compli- 
ance, which ACS NSQIP does not currently do. 
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